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ABSTRACT 



A major challenge facing state policy makers who are demanding 
a high level of accountability in education is to design an 
accountability mechanism to track the progress of education reform. 
This article examines the structure and uses of eight exemplary 
approaches to accountability to find out what makes each mechanism 
work. Emerging from the analysis is a common set of design 
characteristics for state accountability mechanisms that form the 
beginnings of a new theory about state governance in education. 
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With the school reform movement of the 1980s, state policy 
makers have made major monetary and political investments in 
education. State budgets feature new money for education; on state 
agendas, improving the quality of education is a high priority. 
As state commitments have increased so have demands for 
accountability. In a recent survey by the National Conference of 
state Legislatures, accountability was identified as a top priority 
of committee chairmen, and when the 1989 state legislative sessions 
opened, more than thirty states had accountability issues scheduled 
for discussion (Pipho, 1989) . 

The challenge facing state policy makers who are demanding a 
high level of accountability is to determine which accountability 
mechanism is best, in what contexts and for what purposes. This 
essay examines a range of exemplary approaches to accountability 
to find out what makes each mechanism work. Emerging from this 
analysis is a common set of design characteristics for 
accountability mechanisms that form the beginnings of a new theory 
about state governance in education. 

An accountability mechanism in education is defined as a body 
or group that uses educational indicators to track the progress of 
education policy — are state education reforms improving schools? 
Its function is to oversee (monitor and evaluate) the performance 
of the education system and to propose needed changes *-.o policy 
makers. Effective accountability mechanisms feed information into 
the policy cycle, improve schools, and according to the National 
Association of state School Boards, should be »»biased toward 
action." Their purpose should be "not just to inform the public 
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on the state of education, but to take significant action" (1988, 
p» 8). 

Most states have assumed primary responsibility for 

accountability and so acv-ountability mechanisms usually are 

established at the state level with access to the state's data 

bases. ^ "Assigning the State primary responsibility [for 

accountability] often capitalizes on the greater technical capacity 

and resources at that level," observed the U.S. Department of 

Education's state Accountability Study Group (1988, p. 16). 

Cibulka (1989) further found in his study of accountability systems 

in Illinois and South Carolina that the state is a critically 

important motivator of local school districts: 

...among South Carolina school officials interviewed, 
virtually every superintendent was able to cite some 
positive change in policy and practice which ensued due 
to the [state] accountability process. . .The central point 
is that the state policy facilitated the likelihood that 
local action would be taken (p. 9) . 

In fact, policy studies have found that state reforms enhance local 

activism,' as districts often respond to state initiatives by adding 

to them (Fuhrman & Elmore, 1990; Verstegen, 1988). 

Background 

State legislatures have ultimate oversight responsibility but 
the responsibility, in practice, often devolves by default to 
legislative policy committees because a broad range of 
accountability mechanisms and their feasibility are not immediately 
known. However, the standard conclusion in research on the 
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oversight process is that oversight is a neglected function largely 
because policy committees prefer spe:.ding time on more politically 
appealing activities. Rosenthal (1981), in a study of state 
legislatures, observed that the political rewards for introducing 
and passing new legislation are greater than those associated with 
monitoring implementation of existing legislation, which is time- 
consuming and less often leads to substantial or concrete results: 
"Legislators try to achieve concrete results, but oversight seldom 
leads to the elimination of governmental programs or to the 
generation of large budgetary savings" (Rosenthal, 1981, p. 119). 
Therefore, during the legislative session, policy development 
activities, which give legislators opportunities to enhance their 
prospects for reelection (Mayhew, 1974), take center stage and 
little time is available for oversight of existing programs. 

state departments of education also have traditionally been 
involved in overseeing the implementation of education policies and 
programs^ For instance, state departments often receive requests 
from legislative policy committees to conduct monitoring or 
evaluation studies. In the 1980s, the popular "top-down" approach 
to education reform has put a squeeze on departments. State 
departments of education have more responsibilities but resources, 
including money and staff, remain fairly constant (Fuhrman, 1989a) . 
Hence, state departments of education, like legislative policy 
committees, usually are overburdened and more interested in things 
other than oversight. 
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PuypQg^ Qt ^ study 

The present study analyzes alternatives to the legislative 
policy coromittee model to expand the range of accountability 
mechanisms available to states for overseeing education. The 
itiechanisms described here also help relieve the squeeze on state 
departments of education. Prior research on educational 
accountability has focused on developing indicators to assess the 
impact of education reform. The present study is distinguished by 
its focus on the mechanism or group who uses the indicator data and 
is responsible for holding school districts accountable. The 
research issues investigated here relate to governance, whereas 
most other studies of accountability focus on measurement issues. 

The discussion that follows is based on an investigation of 
exemplary state accountability mechanisms. The intent was to 
include mechanisms that highlight the range of diverse approaches 
for overseeing education, apart from the traditional legislative 
policy (education) committee. The selection process was based 
mostly on nominations by national associations oZ state of f icialc..^ 
Mechanisms were nominated based on perceptionss of superior 
performance. In addition, the chairperson and research staff for 
the U.S. Department of Educati.on's state Accountability Study Group 
were consulted because of their familiarity with state systems. 
After telephone interviews, seven mechanisms in seven different 
states were selected for further study, it is important to note 
that the mechanisms included here do not represent the universe. 
Other state boards of education, for example, have accountal ility 
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systems; Connecticut's is simply the best, as judged by the 
nominators* Innovative approaches outside the education arena 
also were considered since monitoring skills are generic"^ and most 
likely transferable, without much difficulty, from one policy area 
to another. The eighth mechanism discussed in this study is one 
such approach: it is emerging and has not yet been tested to 
monitor the performance of education reform. 

Data for this study were collected primarily from telephone 
interviews with staff for the accountability mechanisms, using a 
structured guide. For each machanism, two to three people were 
interviewed for sixty to ninety minutes. 

The first section of this essay analyzes the organization and 
design of different types of accountability mechanisms and 
highlights the various ways in which states approach 
accountability. The second section discusses how the approaches 
have worked by examining the uses and consequences of the 
accountability mechanisms ~ what difference have they made? The 
essay concludes by proposing a set of critical components for state 
accountability mechanisms based on the positive experiences in the 
states studied. 

Models af Accountability Mechanisms 

All eight state accountability mechanisms in this study have 
similar purposes: to find out how education reforms are working, 
and to improve schools and education. Their process for conducting 
accountability studies or evaluations also is remarkably similar. 
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First, the mechanism at the request of, or at least in consultation 
with, policy makers selects an evaluation topic. Next, the 
evaluation study is designed and conducted, either by the 
mechanism, itself, or by staff under the mechanism's supervision. 
The last step is issuing the final report. Typically, the final 
report is issued as a product of the accountability mechanism 
regardless of the mechanism's involvement in conducting the 
evaluation. 

Although purposes and procedures are similar, those in control 
of the mechanisms vary considerably. Some of the accountability 
mechanisms are housed in the legislative branch; one in the 
executive branch; and several use resources in the private sector. 
The mechanisms also differ in terms of the "directors" of 
accountability: they vary from general ists to functional 
specialists (evaluation experts) to policy specialists (education 
experts) . For purposes of discussion, the mechanisms are grouped 
under four types of models, depending on who is in charge: the 
legislative oversight committee model, the executive branch model, 
the partnership model, and the third-party model. One mechanism 
under the partnership model joins the education and business 
communities; the other couples state and local systems. Mechanisms 
under the third-party model involve management consultants from 
universities or private accounting firms. 

Prior research on legislative oversight (Bibby, 1968; Kaiser, 
1977; Ogul, 1976; Rosenthal, 1981 ) has identified a number of 
institutional or structural factors associated with successful 
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oversight including: 

o Legal authority — This is the most obvious of the 
factors since without authority oversight efforts are 
minimal . 

o status of the oversight body — The higher the status of 
the members of the oversight body, the more opportunities 
members will have to influence oversight. Usually the 
higher a member's status, the greater the access to 
resources needed to conduct oversight, 
o Member priorities — Members of oversight bodies are 
faced with a variety of obligations. How important the 
member perceives an area determines how much attention 
that area receives. Therefore, if oversight is a top 
priority, it will receive a great deal of attention, 
o Staff resources — An adequate staff and the willingness 
to use it are vital factors for effective oversight; 
without them oversight is virtually impossible. 
These oversight factors provide a useful framework for analyzing 
the strengths of the accountability mechanisms in this study. All 
eight mechanisms, not surprisingly, have legal authority or a 
statutory basis for oversight activity; the other oversight factors 
are stressed more in some states than in others. Table 1 
summarizes features relating to these factors for all eight 
accountability mechanisms. 
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Insert Table 1 about here 



Legislative Oversight Committee Model 
While three exemplary mechanisms that are part of the 
legislative branch are all joint house-senate oversight committees, 
they involve the legislature's "education experts" (members who 
serve on education committees) to varying dec^rees. The Select 
Committee on Education in Tennessee combines eight education 
experts with eight finance committee members. In Texas, 
legislative leaders in the house and senate (e.g., speaker of the 
house, lieutenant governor, finance committee chairs) comprise the 
majority of the ten-member Legislative Education Board, with only 
two seats reserved for education experts — the chairs of the two 
education committees. 

The third mechanism in this group exploits program evaluation 
and auditing committees commonly found in many state legislatures. 
As described here, the mechanism is based on Virginia's Joint 
Legislative Audit and Review Commission but has not yet been tested 
to oversee education reform on a regular basis. ^ Virginia's Joint 
Legislative Audit and Review Commission, like the Legislative 
Leader Committee in Texas, is controlled by generalists. in 
Virginia, however, members typically stay on until they leave the 
legislature (some have served since the commission's inception in 
1973), so many become j|g ianto evaluation specialists during their 
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tenure. (Committee staff have expertise in evaluation but 
typically are policy generalists. ) The committee, by law, must 
draw a portion of its membership (about 50 per cent) from the House 
Appropriations and Senate Finance CommitteeG. "The commission's 
direct link to the two money committees," one staff member 
observed, "makes the commission a prestigious assignment and the 
most powerful legislators want to serve." The comraittee's 
membership, like the mechanism in Texas, tJ.-aditionally includes 
ranking house and senate leadership (chairs of the fiscal 
committees, majority/minority leaders, president pro tem) . 

Not only does the status of the committee facilitate 
membership recruitment, but it also is a factor associated with 
oversight activity. Bibby (1968) and Ogul (1976) in their studies 
of Congress found that oversight is more likely to occur when 
committee assignments are sought by legislators with more than 
average competence and when members have seniority and 
training/ experience in Congress. The higher a member's status, the 
greater the access to resources needed to conduct oversight. 
Furthermore, legislators seeking assignments to oversight 
committees most likely regard the enterprise as important and so 
are willing to devote resources to oversight activities. 

A second strength in the design of the mechanisms following 
the legislative oversight committee model is their limited scope 
of authority. Prior research on state legislatures (Wohlstetter, 
1989) shows that oversight is not neglected by legislative 
committees when oversight is among the committee's stated 
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priorities. The mission statement relieves members of their policy 
development responsibilities and, in effect, gives them permissit^n 
to spend time monitoring and, equally important, to claim credit 
with their constituencies for such activities. The accountability 
mechanisms established in Tennessee and Texas in the 1980s as part 
of the state's major education reform act have as their primary 
mission the oversight of education. Since the Tennessee and Texas 
oversight coi.imittees do not raise legislation, members are not 
distracted by immediate concerns as standing committees are. 

The ev&luat ion-audit committee is limited by function to 
oversight — the committee does not have bill raising authority, 
for example — but is responsible for overseeing seven different 
areas of state government, only one of which is education. In a 
recent study of education committees in eleven states. Rich Jones 
of NCSL warns: "Legislators must realize that a committee can 
afford to pay attention to only a few programs a year (1988, p. 
106)." ^n Virginia, the committee's agenda is limited, subject- 
wise, by a review cycle that focuses committee activity on 
particular policy areas. For example, when education was evaluated 
over a two-and-a-half to three-year period, some (not all) of the 
other policy areas also were reviewed. 

Studies of Congress by Bibby (1968) and Ogul (1976) suggest 
that oversight its more likely to occur when staff resources are 
sufficient in number and expertise, and when committee funds are 
adequate, state government expert, Alan Rosenthal, agrees and adds 
that the ".egislative oversight model is best suited to state 
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contexts where committee staff and staff leadership are strong and 

legislative support, in terms of committee membership and funding, 

are strong as well. On all thes^ dimensions, the evaluation-audit 

committee alternative is strongest among the three legislative 

oversight committee mechanisms, having an independent operating 

budget, its own staff (with experience and advanced degrees) and 

strong staff leadership. 

The staff in Virginia, moreover, are evaluation experts who 

pride themselves on independence from the legislature as well as 

the executive bran9h, as one staff member explains: 

Division chiefs and the director are the only staff that 
interact with legislative members. Work is done totally 
outside the view of legislative members. The staff receive 
input at the front -end, and typically are very secretive about 
their work, having no contact with the legislature until the 
final briefing. 

Although independence enhances credibility, staff without 
substantive expertise may be unfamiliar with the issues being 
evaluated, so committee staff must educate themselves about each 
program they review which tends to lengthen the study process. In 
Virginia, committee staff typically spend two to three months at 
the beginning of the study process reading documents from both the 
academic and professional literatures and conducting 200 to 300 
interviews. Another strategy is to have education committee 
members who know the problf:m areas define the focus of the 
evaluation, since it is these committee members who field 
complaints by constituents and interest groups. Their 
participation would stir legislators' interest in the evaluation. 
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which would later benefit the program evaluation and audit 
committees when they roust work through standing coromittees to enact 
legislative changes. Virginia's Joint Legislative Audit and Review 
Commission system provides for subcommittees with members of 
substantive standing committees, such as education, involved in 
selecting and scheduling topics, and coordinating studies with the 
standing committees that have jurisdiction over the subjects under 
review. 

Having access to staff, as in Tennessee,^ probably is not 
sufficient for accountability mechanisms. According to Anderson 
and Elliott (1986) , who studied oversight in the South Carolina 
General Assembly, a committee needs staff that are answerable only 
to the committee to avoid conflicting loyalties to groups other 
than the committee. In no case should the committee's work receive 
secondary priority. Furthermore, without assigned staff, committee 
members have no one to press or prod them to focus on oversight. 

In addition to providing resources for oversight, 
appropriating staff and money establishes a contractual 
relationship between the legislature and the mechanism. Contracts 
can be implied such as in Virginia where the legislature relies on 
one of its own committees to serve as the accountability mechanism, 
issuing in effect an in-house contract. Other times the contract 
is explicit as when the legislature enlists an independent 
consultant (see "the third-party model," p. 21). Regardless of 
type, a contract helps establish an audience for monitoring 
information. The government has a vested interest in what the 
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mechanism produces since they have committed to a contract, 
allocated resources, and paid in advance for the services. 
Furthermore, contracts often conclude with public hearings or 
meetings which better enable the mechanisms to feed information 
into the policy cycle by providing them with a public forum for 
presenting their findings. 

Executive Branch Model 

Accountability mechanisms housed in the executive branch are 
more easily subject to charges of "conflicts of interest" than 
mechanisms that are independent of implementing agencies. The 
accountability mechanism described in this section illustrates how 
the executive branch can retain some independence and effectively 
monitor education reform. 

The state of Connecticut vests responsibility for 
accountability with its state board of education. The nine-member 
board is appointed by the governor, anu according to one board 
member; "Being appointed makes it easier for us to stay out of 
politics. We don't have campaign promises to fulfill or axes to 
grind." The board keeps its distance from the legislature, too. 
The commissioner carries the board's agenda to the legislature as 
well as testifies before committees on the board's behalf. 

The board likewise maintains independence from the education 
community. Board members typically are not professional educators 
or from education constituencies, m 1988-89, the board included: 
three former legislators; three representatives from the private 
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sector (an attorney, businessman, and physician) ; and three former 
local school board members. As one board member explains. "The 
state board views itself as representative of the general citizenvy 
who holds educators accountable." Indeed, the board takes 
positions that often are at odds with the teachers unions' views. 
In the recent past, the board set high (not minimum) standards for 
students and teachers, and also rejected a proposal to create a 
separate teacher standards board that unions strongly supported. 

The Connecticut state Board of Education derives oversight 
powers from its statutorily mandated mission that charges the board 
with "promoting che improvement of education in the state... and 
reporting on the condition of the public schools and the quality 
of insstiAiction. " Oversight, however, is not the sole or even a 
primary mission of the state board. The board conducts oversight 
in addition to its other responsibilities of setting policy and 
standards, and providing leadership and services to local school 
districts. 

Prior research has shown that political variables, such as the 
balance of power between the state and local districts, influence 
the development, implementation and monitoring of education reform 
(Fuho^an, 1989b; Wohlstetter, 1989). When states traditionally 
have been leaders in education policy, state control of education 
through legislative mandates and oversight is accepted practice. 
Conversely, in Connecticut where local control traditions are 
strong, a specially-created, state-controlled accountability 
mechanism (as in Tennessee or Texas) most likely would not be 
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politically viable because of the implied loss of local discretion. 

A major contributor to the state board's success as an 
accountability mechanism is the wealth of the state, which 
Connecticut uses partly to enhance staff professionalism among 
state agencies. The department of education's division of retiearch 
and evaluation, which sewes as staff to the board, boasts 70 
professionals, status (a division, not a unit), and sophistication 
(many staff hold doctorates; the division chief, a Ph.D. from 
Stanford) . Indeed, the state board of education mechanism seems 
best suited to a context where strong local control norms inhibit 
the creation of an independent accountability mechanism, but where 
the state has sufficient wealth to beef up department resources 
specifically for accountability activities. 

The governance structure in Connecticut also is very 
compatible and supportive of the state board as overseer. The 
board appoints the commissioner of education and the commissioner 
manages the state department of education. The state board, in 
effect, functions as a board of trustees for the state department 
of education, with the commissioner in the role of CEO. A 
professional educator, the commissioner seirves as staff to the 
board. Members rely on him to formulate ideas and bring policy 
options to them. Also, since the commissioner holds press 
conferences and has a good relationship with the media, the board 
reiies on him "to gcc the word out — to give thj public the sense 
the board is monitoring the condition of education.*' Communicating 
findings beyond the education community also keeps education reform 

18 

ERIC 



16 

salient on the state's agenda. 

In the executive branch model, unlike many accountability 
mechanisms, final reports are issued as products of the staff 
(state department of education) , not the mechanism. According to 
board members, this arrangement makes practical sense since the 
accountability unit is located near, and has direct access to, the 
data bases from which evaluations and recommendations are 
developed, which avoids duplication of efforts and encourages 
reliable and responsible reporting. Board members further argue 
that problems of subjectivity and errors in data collection are 
tempered by information reports to the public which act as checks 
on the system. 

Partnership Model 

While other models feature accountability directors in one 
branch of government at the state level, the two variations of the 
partnership model rely on pairings. South Carolina's 

accountability mechanism joins the executive and legislative 
branches, along with the education and business communities; New 
York's joins the state and local governments. 

The South Carolina mechanism includes three different 
components, all of which were established by the state's iv.ajor 
education reform act of 1984 and amended in 1989 by the "Target 
2000 — School Reform for the Next Decade" Act. One component, the 
Select Committee to Oversee the Implementation of the Education 
Improvement Act, is under the direction of the legislature and is 
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composed mostly of legislative leaders, as in Texas. The executive 
branch also participates with the governor, lieutenant governor and 
state superintendent of education serving on the select committee. 
The second component is the Business-Education Subcommittee. The 
group is a subcommittee and "the working arm" of the Business- 
Education Partnership for Excellence in Education whose original 
members were appointed by the governor in the early 1980s to 
develop a rlan for education reform, which became the South 
Carolina Education Improvement Act of 1984. Elected by the 
partnership, the subcommittee's membership must be composed of ten 
business and civic leaders, six educators, and four legislators. 
The third component of South Carolina's mechanism is the Division 
of Public Accountability established in the state department of 
education as part of the 1984 reform package. 

The primary mission for all three components is oversight of 
the Educacion Improvement Act (EIA) . Although the mission is 
shared, funct;ions are distinct. The division of public 
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accountability, which has data capabilities and access to the 
department of education's data bank, is responsible for preparing 
an annual assessment report on student performance ( What is the 
Pgnny ^uyi^g las south Carolina? ^ , for the subcommittee and the 
select committee. The subcommittee functions as an independent 
monitoring body representing the public. For example, the 
subcommittee serves as an advisory committee to the division of 
public accountability and is responsible for providing citizens - 
- this includes the Business-Education Partnership which by law 
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meets quarterly — and the -state legislature with an evaluation of 

the progress of EIA*s itnpleinentation. The select committee is the 

EIA oversight committee in the legislature and its annual report 

keeps members well-informed: 

The annual report includes a review of all oversight 
activities conducted by the Committee, and reports findings 
and conclusions resulting from program reviews, studies and 
evaluations. Specific program-related issues for which 
recommendations are proposed by the Committee are also 
included for consideration (Research Director, EIA Select 
Committee) . 

The select committee also functions as a clearinghouse for all 
proposed EIA changes: legislative amendments and budget proposals 
are considered by standing committees only after ths select 
committee's recommendation or approval. 

Linkages to various constituencies, both within and outside 
government, have been a key strength of South Carolina's mechanism, 
according to the executive director of the subcommittee. Research 
on implementation suggests that education reforms need time to 
mature, ard that changes in practice and performance lag behind 
electoral cycles (Elmore & McLaughlin, 1988) . As a result, the 
original coalitions that work to support education reform may be 
broken up by elections before the reforms mature. The newly- 
elected leaders, moreover, may not have the same commitment to the 
reforms as their predecessors (Thomas, 1987). South Carolina's 
diverse membership, which includes business and -.ommunity 
representatives, educators, the governor and legislators, helps 
sustain momentum for the reforms: the more committed reformers 
prod the less committed to stay on the band\'agon. According to the 
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executive director, the mcinbership also is helpful in "getting the 
word out" to various constituencies about educational progress, and 
such communication has been instrumental in retaining a broad base 
of support for the reforms six years after their enactment. 

New York's accountability mechanism pairs state education 
specialists in the executive branch — the Board of Regents 
(comparable to a state board of education) , the commissioner of 
education and the department of education — with local school 
districts. The mechanism, which began a two-year pilot test during 
Fall 1989, was established earlier that year by the Board of 
Regents, New York's chief education policy maker^, as part of its 
Excellence and Accountability Program (EAP) . According to one of 
its creators, a key principle of the program is that accountability 
should be primarily a local function, with state intervention only 
when a breakdown in the system occurs. 

The primary role of the partners at the state level is to set 
broad policy guidelines and minimum achievement standards for local 
districts. In contrast to state-controlled mechanisms in the study 
sample. New York's establishes individual schools as the locus of 
accountability based on the idea that local districts should be 
accountable foremost to local constituencies.^ Individual schools 
will have flexibility to create their own educational programs and 
monitor outcomes. Local districts, in effect, will be allowed to 
operate their own accountability systems, provided that school 
performance meets the minimum standards set by the Board of 
Regents. The state department of education's Office of Quality 
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Assurance will play mainly a supportive role, offering technical 
assistance to help districts create programs and develop standards 
of excellence for their individual educational plans. 

The commissioner's role with the accountability mechanism will 
be reactive in that the commissioner will step in only in response 
to a local district's request or because of poor performance. The 
powers vested in the commissioner are significant. For example, 
the commissioner has the power to deregulate school districts on 
an individual basis. According to EAP, any district can request 
a variance to waive, for instance, a reporting requirement if the 
district can demonstrate that what it wants is better than what the 
state prescribes. The commissioner also has the authority to 
intervene when local school districts fall below minimum state 
achievement standards and continue to decline for three years. For 
poor performing schools, the commissioner can revoke their state 
license, force the school to shut down, and takeover the 
responsibility of providing a quality education for the students. 

EAP was designed on the assumption that a fully and accurately 
informed public is an essential component of accountability. A key 
strength of the mechanism, therefore, is the extent to which 
monitoring activities and results are communicated to multiple 
constituencies. state guid;ilines for the pilot test require 
schools to solicit input from community members (e.g., parents, 
business representatives, cultural institutions) in developing 
their individual educational plans, and also to publish annual 
reports and present the results (district-wide and school-by- 
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school) at local public meetings. With an informed public, the 
reforms benefit from volunteer monitors (the public) who track 
implementation and sound alarms when the reforms are not working 
or need improving (Wohlstetter, 1990) . 

At the state level, the Office of Quality Assurance will 
develop and maintain a second data base for each district, in 
addition to the district's own, composed of information contributed 
by every school district. This state data base will be used to 
compare districts across the state and to develop the minimum 
achievement standards with which all districts must comply. 
Finally, with the benefit of its own public relations unit, the 
Board of Regents plans to communicate results to various 
constituencies through published reports and public hearings held 
throughout the state about twenty times a year. From the 
literature on oversight (see p. 7) , it is also likely that 
publicity by the Board, whose members enjoy considerable statewide 
visibility and status, will help elevate oversight on local 
district agendas. 

Third-Party Model 

Conflicts of interest are a problem for accountability 
mechanisms in education. Often program administrators — state 
departments of education and local school districts — are also 
responsible for collecting evaluation data. David Savage of the 
Lqs hnqeX^s Tims sees the diial role of implementor and evaluator 
affecting reliabi."" ity: 
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When something called the National Assessment of Educational 
Progress puts out a report saying that students today are 
reading better., a reporter and his/her editors tend to view 
that finding as reliable. When the Los Angeles school 
district puts out a report saying that its students are faring 
much better, the reporters and editors are at least mildly 
skeptical (1989, p. 22). 

To solve the problem, Donald Thomas, former director of the 
South Carolina Education Department's Division of Public 
Accountability, proposes contracting with a third-party to achieve 
effective accountability. 

I think it is just a given that agencies of government have 
a tendency to serve themselves rather than the clients they 
were established to serve... I think once the accountability 
portion of any agency is subsumed by that agency the chances 
for abuse are far greater than if the accountability program 
is an independent process (Thomas, 1987} . 

The fourth group of accountability mechanisms features 

subcontracting with third parties to evaluate education reforms. 

One mechanism relies on a university consortium to track progress 

while the other mechanism utilizes performance auditors in 

independent accounting firms. 

The 'university consortium mechanism originated in 1983 with 
PACE — Policy Analysis for California Education. There are now 
about 20 states that use similar mechanisms, to varying degrees, 
to evaluate school reforms and state education practices. 

PACE'S priority or primary mission is not oversight. In fact, 
program evaluation is only one of the consortium's five, self- 
assigned tasks. The other four are: 1) collecting and 
distributing objective information about the conditions of 
education in California; 2) analyzing state education policy issues 
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and the policy environment; 3) providing technical support to 
policy makers; and 4) facilitating discussion education issues. 
To date, PACE has conducted one large-scale evaluation of 
California's major education reform, Senate Bill 813, which was 
enacted in 1983. 

The directors of PACE, most of whom are professors at schools 
of education, are education exports and closely tied to 
California's education and political communities. "The four PACE 
directors have over one hundred years of combined experience 
working with state education and political leaders," reports one 
director. This experience enhances PACE'S status and credibility 
within the education community. Moreover, the university-base 
helps PACE project an image of objectivity and neutrality, as 
Savage of the Los Anaeles Times observes: 

Michael Kirst and Jim Guthrie [directors of PACE] can get more 
attention for their reports because they are professors at 
Stanford and Berkeley, respectively. if they both taught 
remedial English at West Los Angeles Community College, their 
findings would be given less credence in the press, solely for 
that reason. When looking at a research report, editors and 
reporters want to know about the reputation of the authors, 
their affiliations and the basis for their findings (1989, pp. 
22-23) . 

Unlike other accountability mechanisms in the sample, PACE 
directors are involved in all aspects of conducting evaluations, 
from selecting topics to data collection, analysis and report 
writing. PACE studies are staffed by education specialists, 
usually faculty and graduate students from the member institutions. 

Another difference with PACE is its independence from the 
5tate: PACE is not empowered by the government to conduct 
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oversight. Unlike many other mechanisms, the consortium is a 
private research organization, not a legislatively-mandated body. 
The tjovernment also plays a small funding role. Although the PACE 
evaluation of Senate Bill 813 was a state contract, most PACE 
studies are self- initiated, not government-authorized. On-going 
financial suppor-t is provided by a private foundation and this 
arrangement enhances PACE'S independence. 

Without a state mandate, PACE relies on personal contacts to 
interest policy makers in its studies. The directors have strong 
personal and professional relationships with all key state policy 
makers, including the state superintendent, the state board and 
state department of education, the governor's office and the 
legislature. According to one director, "The contacts help us from 
outside the system feed important study findings into the policy 
process." PACE also has a "superb relationship with the media" and 
has developed somewhat of an expertise in dealing with the press. 
For example, press conferences are held for all major reports to 
comment on the findings. The publicity, of course, adds visibility 
to the issues and puts pressure on the policy makers to act or, at 
least, pay attention. 

The last accountability mechanism in the sample creates 
ex. licit government contracts with performance auditors in the 
private sector. This mechanism has not yet been tested in the field 
of education; however, the concept is not new. In fact, most Big- 
Eight accounting firms have perfoimance audit divisions and some 
smaller firms specialize in performance audits. 
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Local school districts in a number of states (e.g., 
California, Maryland, Virginia) currently are required by statute 
to undergo independent financial audits. The performance audit 
mechanism, proposed here, expands current audit requirements into 
the performance arena. This idea also is not new. State mandates 
for performance audits already are used in the field of 
transportation, for example, where requirements call for 
independent performance audits of service providers every three 

a 

years. Like other mechanisms in this study, performance audits of 
local school districts would focus on the extent to which program 
results have been achieved, with an aim toward improving the 
education system. 

One strength of the mechanism is its independence from the 
implementors of education reform, namely departments of education 
and local school districts. As evaluation specialists, performance 
auditors arguably would have fewer conflicts of interest than 
mechanisii[is with ties to the education community. 

Despite its independence, the performance auditor mechanism 
still would depend on monitoring data typically collected by 
implementing agencies. (Even PACE, which does a substantial amount 
of original data collection, relies heavily on state department of 
education data.) Following the transit industry model, the 
credibility of the audit could be enhanced by having performance 
auditors first focus on compliance questions — is the school 
district collecting and reporting statistics appropriately? — and 
then focus on management issues including trends over a three-year 
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period and recommendations for administrative and legislative 
changes . 

Another unique feature of this mechanism would be the 
specificity of its legislative mandate. In the transit industiy, 
authorizing legislation specifies data elements that transit 
operators must collect (e.g. vehicle miles, in-service mechanical 
failures), and assessment measures (ratios of data elements) that 
auditors must use to evalua' performance (e.g., number of vehicle 
miles per in-service faixures) , in addition to a three-year 
evaluation cycle. 

A similar approach to education may be just around the corner. 
Twenty-three states already have in place indicator systems that 
offer a smorgasbord of data elements, according to a 1987 survey 
by the Council of Chief state School Officers. And within the 
research community, attention is focused on creating new 
educational quality indicators to assess the impact of reform (see, 
for example, Kaagan & Coley, 1989 and Shavelson et al., 1987). 

HfiSS and Consequences af Accountability Mechanisms 

Weiss (1989), in her study of congressional committees as 
users of analytic information, developed "the categories of use" 
method to analyze the impact of congressional committees. As 
overseers evaluating performance, accountability mechanisms are, 
by definition, users of analytic information. This information, 
according to Weiss (p. 425) , can serve four main functions: 

o Support — certifies that the pre-existing position is 
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right 

o Warning — signals that a problem is (or is not) severe 
o Guidance — indicates better alternatives 
o Enlightenment — offers new constructs, new ways of 

thinking about issues 
Respondents' observations about how the accountability mechanisms 
actually work suggest they can serve any of the four functions but 
tend to concentrate on the first three, as Weiss found with the 
committees in Congress, in addition, each mechanism in the sample 
has a dominant function, one that is performed almost to the 
exclusion of the others. Some mechanisms apparently serve one 
function better than others. Table 2 provides a summary of the 
functions of accountability mechanisms, based on responses to the 
question, "what difference — symbolic and tangible — has the 
group made to the state generally and to education reform, in 
particular?"^ 



Insert Table 2 about here 



Across the mechanisms, the most common function is guidance 
where monitoring information clarified legislative intent, or led 
to legislative provisions or amendments. in all instances cited 
by respondents, irformation use focused on detailed issues rather 
than broad policy strategy. The mechanisF' eiso tended to have 
strong connections with educaticn committees, since following up 
on tha mechanism's advice usually required legislative action. 
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Several mechanisms in the study sample combined the warning 
and guidance functions. For example, in South Carolina when the 
state department of education's Division of Public Accountability 
discovered that 45 percent of the state's high school students 
failed all three parts of the exit exam required for graduation, 
the division recommeiided to the legislature that a priority order 
be established for serving compensatory and remedial students so 
that those who needed instruction most would be served first. As 
a result, the EIA Compensatory and Remedial Program was redirected 
from the elementary schools, where the focus had been on early 
detection, to providing high school students who failed one, two 
or three parts of the exit exam with remedial instruction. 

The function that was next in frequency is support. some 

respondents cited cases where accountability mechanisms helped 

strengthen the coalitions supporting education reform. In Texas, 

where the legislature meets only every other year, the LEB has been 

instrumental "in keeping education issues on the front burner and 

alive with the public and the press." in South Carolina, where 

business community representatives serve with educators and 

legislators on the Business-Education Subcommittee, the mechanism 

has helped retain significant business interest in education. 

"Business has been a major partner in selling education reform," 

commented the state superintendent of education. The mechanism 

also has fostered healthy working relationships among participants. 

•There were times, early on, when there would be a c.E.O. 
sitting next to a teacher, and they would barely speak to one 
another, • said Terry Peterson, the subcommittee's executive 
director. "Now, they'll sit by each other and talk' (Flax, 
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1989) . 

Infonning the public about the progress of reforms; also can 
lend support by reinforcing reform advocates' confidence in their 
stand and by weakening opponents' support. The PACE study of 
Senate Bill 813 in California purposefully did not address whether 
or not the reforms were improving education. Entitled How state 
Ec^ucatjpyi Pj?fffrm £an Improve Secondar y Schools , the PACE study 
sampled only schools that were improving and documented the 
reform's benefits (Odden & Marsh, 1988). PACE findings featured 
in some 40 or 50 newspapers invigorated public interest in 
education in California and also provided ammunition to reform 
advocates' policy debates at the state level. 

The final function is enlightenment, in which mechanisms help 
change the way people think about issues. one substantive 
illustration mentioned was in Tennessee with the issue of equity. 
Through its analysis of monitoring data, the Select Oversight 
Committee on Education discovered significant differences between 
small-rural and big-city school districts regarding programs and 
funding. The oversight committee's report received considerable 
press coverage, and rural districts learned for the first time that 
programs being implemented in the big cities were unavailable in 
their districts. Since then a lawsuit on behalf of small school 
systems has been filed against the state to remedy the inequities 
between Tennessee's rural and big-city districts. 
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CQnPXUSiQns ^ Implications 

As mentioned at the outset, the purpose of this study was to 
expand the range of organisational alternatives available to states 
for overseeing education reform. The underlying rationale was that 
if accountability mechanisms effectively monitored performance and 
proposed needed changes to policy maksrs, educational practice 
would improve. 

Emerging from the analysis of the eight exemplary 
accountability mechanisms is a set of criteria that apply to the 
design of any mechanism, regardless of the state governance 
structure for education or political context. The criteria propose 
that any effective accountability mechanism should: 

(1) Empowered M state Government . Accountability mechanisms 
that receive a government contract to monitor education reform 
employ the government as an interested audience for their 
reports. The government has committed resources (money and 
staff) in advance and will want to know the return on their 
investment. Furthermore, presenting findings in public 
forums, such as conunittee hearings or meetings, enhances the 
likelihood that monitoring information will impact policy. 

(2) Hays Monitoring or Oversight as a Primary Mission . Specially- 
created accountability mechanisms whose main task is oversight 
spend more time monitoring than re-tooled policy making bodies 
that have simply added oversight to their other 
responsibilities. Specially-created mechanisms attract 
members who consider the oversight vmterprise important and 
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members can claim credit with their constituencies for such 
activities when oversight is the primary mission. 

(3) £g Independent from Implementors . Accountability mechanisms 
that remain separate from implementors of the reforms do not 
evaluate their own performance and, thus, at least avoid the 
perception of "conflicts of interest." The mechanism's image 
of objectivity and neutrality, moreover, enhances the 
credibility of their reports. 

(4) Hav^ strong Relationships with Other Policv Actors With 
Leaders Outside Government . Accountability mechanisms with 
members who represent various constituertcies help sustain 
momentum for the reforms by expanding interest beyond only a 
few groups. Strong connections with state policy makers also 
increases the likelihood that monitoring information will be 
acted upon since following up on the mechanism's advice often 
requires government action. 

(5) Comrpunicate Findings %q Multiple Constituencies . 
Accountability mechanisms that actively publicize monitoring 
results help stimulate and maintain interest in education 
reform. Publicity also puts pressure on policy makers to act 
or, at least, pay attention to the reforms. 

Table 3 indicates the extent to which the mechanisms in the study 
sample possess these five characteristics. 



Insert Table 3 about here 



34 



32 

Taken together these design elements form the beginnings of a new 
theory about state governance in education that can guide policy 
makers decisions' obout who should be responsible for 
accountability and how they should be organized. 

Current events indicate that states increasingly are deciding 
to deregulate high-achieving school districts and takeover 
"academically bankrupt" districts. Accountability wechanisms 
control the information from which such decisions are made. This 
preliminary study suggc cs that different types of accountability 
mechanisms are suited to various contexts and purposes; none of the 
eight accountability mechanisms is best. Further research is 
needed to elaborate on the capabilities of each mechaii 
including the relationship between the type of mechanism an-i 
particular accountability activities undertaken, and the effects 
of such activities on education policy and practice. 
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Table 1. 

EfiX £eatmeB fi£ Accountability Hfifiilifiillififflfi 



HQdfilfi & YarAationg composition 



Legiglative oversight 

£smi*-tss HQdfil 

Sdticatlon-Finance 

Committee 

(Tennessee) 



16 legislators. 8 from 
education committees & 
8 from finance 
committees. 



Legislative Leader 
Committee (Texan) 



10 legislators. 
Majority are legis- 
lative leaders. Also 
includes house & 
senate education 
committee chairs. 



£valuation-Audit 
Committee 
(Virginia's Joint 
Legislative Audit 
& Review Commission) 



14 legislators. >' must 
be members of house & 
senate finance 
committees. 
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fiCOCe Ql Authority 



Staffing 



Single policy area. 



Single policy area. 



No independent 
staff. Use stand- 
ing committee 
staff & Speaker's 
staff. 

1990 appropriations 
included separate 
staff for LEB. 



7 areas of state 27 research staff. 

government: education, 

enterprises, gene; .1 

government, individual 

& family services, 

justice, resource & 

economic development, 

transportation. 
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Table 1, (cont*d) • 



HQd&U k Varifttions 



Scope i2£ Authority 



SfcaffiLng 



Executive fiiAQfib 
Model 

State Board of 

Education 

(Connecticut) 



9 members appointed 
by governor. 



Single policy area. 



No independent 
staff. Use Commis- 
sioner's staff & 
SDE's division of 
research & evalua- 
tion (70 staff) . 



Partnership Model 

Education-Business 
Partnership (South 
Carolina) 



State-Local 
Partnership 
(New York) 



12 on Select Commit- 
tee & 20 on Business- 
Education Subcommittee. 
Members include governor, 
SPI, legislators & repre- 
sentatives from business 
& education communities. 



Single policy area. 



15 on Board of Regents, 
appointed by governor, 
& individual school 
districts. 



Single policy area< 



a) Division of Pub- 
lic Accountability 
(SDE) : 11.5 PTE. 

b) Select Committee: 
1 FT director 

c) Business-Education 
Subcommittee: 1 PT 
director & 1 PT 
administrative 
assistant. 

a) Board of Regents: 
Commissioner & SDE's 
Office of Quality 
Assurance. 

b) Individual School 
Districts: staffing 
at the discretion of 
local districts. 
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Table* If (cont'd). 





Models & Variafcions 


Composition 


Scope of Authority 


staffing 


Third-Party Model 








University 
Consortium (Policy 
Analysis for 
California 
Education) 


4 professors from 
public & private 
schools of education. 


Single policy area. 


Faculty & graduate 
students from 
member institu- 
tions. Ranges from 
6 to 10. 



Performance Performance auditors Multiple policy areas. Subcontracts with 

Auditors with private accounting performance auditors 

firms. in private accounting 

firms. Performance 
audit teams range 
from 2 to 10, 
depending on number 
of service provi- 
ders evaluated. 
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Table 2. 

Functions fificxed B;^ Accountability Mechanisms 



Hdd£l£ k Variations Guidance Su pport Warning Enlightenment 

Legislative Oversight 
Committee Hodel 

Education-Finance 
Committee 

(Tennessee) D* X 

Legislative Leader 

Committee (Texas) D X 

Evaluation-Audit 
Committee 
(Virginia's Joint 
Legislative Audit 

& Review Commission) D D 



Executive fiiancb 

state Board of 

Education 

(Connecticut) 



Partnership Mfidfii 

Education-Business 
Partnership (South 
Carolina) 



42 



ERIC 



Table 2, (cont'd) . 



Hfidfils k Yariations 



Guidance Su pport Warning Enlightenment 



Xhiid £&cty Hsdel 

University 
Consortium (Policy 
Analysis for 
California 
Education) 

Performance 
Auditors 



* D denotes dominant function. 
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Table 3. 

Ratings tox. AcgQwntability MechaniBBs sm Critical components 



Hodelfi A Variations 



fifi Eapowered 
Government 



flaxfi Monitoring 
QJL Oversight 
AS A Primary 

HiBsion 



Lgaislative Overaight 
rni—i«-i-oo Model 

Education-Finance Moderate 

Committee 

(Tennessee) 

Legislative Leader Moderate 
Committee (Texas) 

Evaluation-Audit High 
Committee 
(Virginia's Joint 
Legislative Audit 
& Review Commission) 



High 

High 
High 



fi£ Independent 
lism Im plementors 



flaxfi £t£fing 

Relationships hIUi 
QthSJL Pfilifiy ActfiCfi 
and Xith Leaders 
Outside Government 



Conmuni- 

fiatfi Zini 

inga to 
Multiple 

Conatitu- 



Moderate 

Moderate 
High 



Moderate 

Moderate 
Low 



Low 



Low 



Moderate 
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Table 3, (cont'd) 



0 



Modelfi k Vaiiatipng 



iBfi Eapowered 

J2X StAtfi 
Government 



flOXfi Monitoring 

fix Qveraight 

AS A Priaary 

Mission 



fifi Independent 
£CfiD Implementors 



Relationships xiUl 

Dtliex fslifiy Actfics 
And Mith LCAders 

Outside Government 



£Atfi fin 
logs to 

Multiple 
Constitu 



Executive fiiAllfb 

state Board of Moderate Moderate Low High Moderate 

Education 

(Connecticut) 



Partnership Model 

Education-Business High High Moderate High High 

Partnership (South 

Carolina) 

State-Local Moderate Moderate Low Moderate High 

Partnership 

(New York) 



UbuLrdrParty Model 



University 
Consortium (Policy 
Analysis for 
California 



Low 



Moderate 



High 



High 



High 
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